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Abstract 

In image classification, merging the opinion of several human experts is very 
important for different tasks such as the evaluation or the training. Indeed, the 
ground truth is rarely known before the scene imaging. We propose here differ- 
ent models in order to fuse the informations given by two or more experts. The 
considered unit for the classification, a small tile of the image, can contain one or 
more kind of the considered classes given by the experts. A second problem that 
we have to take into account, is the amount of certainty of the expert has for each 
pixel of the tile. In order to solve these problems we define five models in the con- 
text of the Dempster-Shafer Theory and in the context of the Dezert-Smarandache 
Theory and we study the possible decisions with these models. 

Keywords: Experts fusion, DST, DSmT, image classification. 

Introduction 

Fusing the opinion of several human experts, also known as the experts fusion problem, 
is an important question in the image classification field and very few studied. Indeed, 
the ground truth is rarely known before the scene has been imaged; consequently, some 
experts have to provide their perception of the images in order to train the classifiers 
(for supervised classifiers), and also to evaluate the image classification. In most of 
the real applications, the experts cannot provide the different classes on the images 
with certitude. Moreover, the difference of experts perceptions can be very large, and 
so many parts of the images have conflicting information. Thereby, only one expert 
reality is not reliable enough, and experts fusion is required. 

Image classification is generally done on a local part of the image (pixel, or most 
of the time on small tiles of e.g. 16 x 16 or 32x32 pixels). Classification methods can 
usually be described into three steps. First, significant features are extracted from these 
tiles. Generally, a second step in necessary in order to reduce these features, because 
they are too numerous. In the third step, these features are given to classification 
algorithms. The particularity in considering small tiles in image classification is that 
sometimes, more than one class can co-exist on a tile. 

An example of such an image classification process is seabed characterization. This 
serves many useful purposes, e.g help the navigation of Autonomous Underwater Ve- 
hicles or provide data to sedimentologists. In such sonar applications, which serve as 
examples throughout the paper, seabed images are obtained with many imperfections 



INFORMATION & SECURITY. An International Journal, Vol. 20, 2006, 1-22 



2 



HUMAN EXPERTS FUSION FOR IMAGE CLASSIFICATION 



(T). Indeed, in order to build images, a huge number of physical data (geometry of the 
device, coordinates of the ship, movements of the sonar, etc.) are taken into account, 
but these data are polluted with a large amount of noises caused by instrumentations. 
In addition, there are some interferences due to the signal traveling on multiple paths 
(reflection on the bottom or surface), due to speckle, and due to fauna and flora. There- 
fore, sonar images have a lot of imperfections such as imprecision and uncertainty; 
thus sediment classification on sonar images is a difficult problem. In this kind of 
applications, the reality is unknown and different experts can propose different clas- 
sifications of the image. Figure Q] exhibits the differences between the interpretation 
and the certainty of two sonar experts trying to differentiate the type of sediment (rock, 
cobbles, sand, ripple, silt) or shadow when the information is invisible. Each color 
corresponds to a kind of sediment and the associated certainty of the expert for this 
sediment expressed in term of sure, moderately sure and not sure. Thus, in order to 
learn an automatic classification algorithm, we must take into account this difference 
and the uncertainty of each expert. For example, how a tile of rock labeled as not sure 
must be taken into account in the learning step of the classifier and how to take into 
account this tile if another expert says that it is sand? Another problem is: how to take 
into account the tiles with more than one sediment? 




Figure 1: Segmentation given by two experts. 



Many fusion theories can be used for the experts fusion in image classification 
such as voting rules possibility theory 0[5]], belief function theory |6l|7). In our 
case, experts can express their certitude on their perception. As a result, probabilities 
theories such as the Bayesian theory or the belief function theory are more adapted. 
Indeed, the possibility theory is more adapted to imitate the imprecise data whereas 
probability-based theories is more adapted to imitate the uncertain data. Of course 
both possibility and probability -based theories can imitate imprecise and uncertain data 
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at the same time, but not so easily. That is why, our choice is conducted on the belief 
function theory, also called the Dempster-Shafer theory (DST) EE) . We can divide the 
fusion approach into four steps: the belief function model, the parameters estimation 
depending on the model (not always necessary), the combination, and the decision. 
The most difficult step is presumably the first one: the belief function model from 
which the other steps follow. 

Moreover, in real applications of image classification, experts conflict can be very 
large, and we have to take into account the heterogeneity of the tiles (more than 
one class can be present on the tile). Consequently, the Dezert-Smarandache Theory 
(DSmT) 00, an extension of the belief function theory, can fit better to our problem of 
image classification if there is conflict. Indeed, considering the space of discernment 
O = {Ci, C2, ■ • • , C n }, where (7, is the hypothesis "the considered unit belongs to 
the class i". In the classical belief function theory, the belief functions, also called the 
basic belief assignments, are defined by a mapping of the power set 2 e onto [0, 1]. The 
power set 2 e is closed under the U operator, and G 2 e . In the extension proposed 
in the DSmT, generalized basic belief assignments are defined by a mapping of the 
hyper-power set D e onto [0, 1], where the hyper-power set D e is closed under both U 
and n operators. Consequently, we can manage finely the conflict of the experts and 
also take into account the tiles with more than one class. 

In the first section, we discuss and present different belief function models based 
on the power set and the hyper power set. These models try to answer our problem. 
We study these models also in the steps of combination and decision of the information 
fusion. These models allow, in a second section, to a general discussion on the differ- 
ence between the DSmT and DST in terms of capacity to represent our problem and in 
terms of decision. Finally, we present an illustration of our proposed experts fusion on 
real sonar images, which represent a particularly uncertain environment. 



1 Our proposed Models 



In this section, we present five models taking into account the possible specificities 
of the application. First, we recall the principles of the DST and DSmT we apply 
here. Then we present a numerical example which illustrates the five proposed models 
presented afterward. The first three models are presented in the context of the DST, the 
fourth model in the context of the DSmT, and the fifth model in both contexts. 
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Theory Bases 

Belief Function Models 

The belief functions or basic belief assignments m are defined by the mapping of the 
power set 2 e onto [0, 1], in the DST, and by the mapping of the hyper-power set D e 
onto [0,1], in the DSmT, with : 

m(0) = 0, (1) 

and 

2 m{X) = 1, (2) 

in the DST, and 

^ m(X) = l, (3) 
xeD e 

in the DSmT, where X is a given tile of the image. 

The equation (Q~|) allows that we assume a closed world (7] [U- We can define the 
belief function with only: 

m(0) > 0, (4) 

and the world is open (9j. In a closed world, we can add one element in order to 
propose an open world. 

These simple conditions in equation (HJ and (0 or (HJ and (O, give a large panel 
of definitions of the belief functions, which is one of the difficulties of the theory. The 
belief functions must therefore be chosen according to the intended application. 

In our case, the space of discernment represents the different kind of sediments 
on sonar images, such as rock, sand, silt, cobble, ripple or shadow (that means no 
sediment information). The experts give their perception and belief according to their 
certainty. For instance, the expert can be moderately sure of his choice when he labels 
one part of the image as belonging to a certain class, and be totally doubtful on another 
part of the image. Moreover, on a considered tile, more than one sediment can be 
present. 

Consequently we have to take into account all these aspects of the applications. In 
order to simplify, we consider only two classes in the following: the rock referred as 
A, and the sand, referred as B. The proposed models can be easily extended, but their 
study is easier to understand with only two classes. 

Hence, on certain tiles, A and B can be present for one or more experts. The 
belief functions have to take into account the certainty given by the experts (referred 
respectively as ca and cb, two numbers in [0, 1]) as well as the proportion of the kind 
of sediment in the tile X (referred as pa and ps, also two numbers in [0, 1]). We have 
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two interpretations of "the expert believes A": it can mean that the expert thinks that 
there is A on X and not B, or it can mean that the expert thinks that there is A on X and 
it can also have B but he does not say anything about it. The first interpretation yields 
that hypotheses A and B are exclusive and with the second they are not exclusive. We 
only study the first case: A and B are exclusive. But on the tile X, the expert can also 
provide A and B, in this case the two propositions "the expert believes A" and "the 
expert believes A and B" are not exclusive. 

Combination rules 

Many combination rules have been proposed these last years in the context of the belief 
function theory ( flTQl [TTl [9] [T^ IBl H31 , etc.). In the context of the DST, the combination 
rule most used today seems to be the conjunctive consensus rule given by [9] for all 

X e 2 e by: 

M 

to w= e n^-ft-). (5) 

Y 1 r\...nY M =x 3=1 

where Yj € 2 e is the response of the expert j, and rrijiYj) the associated belief func- 
tion. 

In the context of the DSmT, the conjunctive consensus rule can be used for all 
X 6 D B and Y £ D . If we want to take the decision only on the elements in 0, some 
rules propose to redistribute the conflict on these elements. The most accomplished 
rule to provide that is the PCR5 given in [ 13 1 for two experts and for X G D & , X ^ 
by: 

mp C B.s{X) = m 12 (X)+ 

^ \ mi (X)+m2(Y) m 2 (X) + m 1 (Y)J ' (6) 

YeD e , 

c(xr\Y)=$ 

where mi2(.) is the conjunctive consensus rule given by the equation (0, 
c(X fl Y) is the conjunctive normal form of X n Y and the denominators are not 
null. We can easily generalize this rule for M experts, for X E D e , 1^(5: 

mpcffi(I) = m{X)+ (7) 

/ A/-1 \ 



E^w 2 E 



M-l 

n y w nx=0 
fe=i 1 

(y„ i(1) ,...,Y„ i(M _ 1) )G(D e ) J 



3=1 



M-l 
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where cr, counts from 1 to M avoiding i: 

/ OiC?) = 3 if 3 < i, (R , 
I ^C?)=i + 1 if J" > i, W 

M-l 

nii(X) + m cr i (i)(^o- i (j)) ^ 0, m * s tne conjunctive consensus rule given by 

i=i 

the equation @. 

The comparison of all the combination rules is not the purpose of this paper. Con- 
sequently, we use here the equation © in the context of the DST and the equation ^} 
in the context of the DSmT. 



Decision rules 

The decision is a difficult task. No measures are able to provide the best decision 
in all the cases. Generally, we consider the maximum of one of the three functions: 
credibility, plausibility, and pignistic probability. 

In the context of the DST, the credibility function is given for all X €2 e by: 

bel(X)= m ( y )' ( 9 > 

The plausibility function is given for all X e 2° by: 

pl(Jf) = m{ y) = bel ( e ) - bel(X c ), (10) 

Y£2 B ,YnX^<D 

where X c is the complementary of X. The pignistic probability, introduced by fPfll . is 
here given for all X e 2°, with X ^ by: 

Generally the maximum of these functions is taken on the elements in 6, but we will 
give the values on all the focal elements. 

In the context of the DSmT the corresponding generalized functions have been 
proposed lfl5l [8ll. The generalized credibility Bel is defined by: 

Bcl(X)= J2 m (Y) (12) 

Y£D X 

The generalized plausibility PI is defined by: 

P1(X) = ( 13 ) 

YeD e ,xnY^£<A 
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The generalized pignistic probability is given for all X 6 D & , with X ^ is defined 
by: 

GPT(X)= £ C *<*" y W ), d4) 

where Cm(X) is the DSm cardinality corresponding to the number of parts of X in 
the Venn diagram of the problem lfl5l [8ll. 

If the credibility function provides a pessimist decision, the plausibility function 
is often too optimist. The pignistic probability is often taken as a compromise. We 
present the three functions for our models. 



Numerical and illustrative example 

Consider two experts providing their opinion on the tile X. The first expert says that 
on tile X there is some rock A with a certainty equal to 0.6. Hence for this first expert 
we have : Pa = 1, Pb = 0, and ca = 0.6. The second expert thinks that there are 
50% of rock and 50% of sand on the considered tile X with a respective certainty of 
0.6 and 0.4. Hence for the second expert we have: pa — 0.5, ps — 0.5, ca — 0.6 and 
cb = 0.4. We illustrate all our proposed models with this numerical exemple. 



Model Mi 

If we consider the space of discernment given by 9 = {A, B}, we can define a belief 
function by: 

if the expert says A: 
( m(A) = c A , 
\ m(A U B) = 1 - c A , 

(15) 

if the expert says B: 
f m(B) = c B , 
\ m(AUB) = l-c B . 

In this case, it is natural to distribute 1 — ca and 1 — cb on AU B which represent the 
ignorance. 

This model takes into account the certainty given by the expert but the space of 
discernment does not consider the possible heterogeneity of the given tile X. Conse- 
quently, we have to add another focal element meaning that there are two classes A and 
B on X. In the context of the Dempster-Shafer theory, we can call this focal element 
C and the space of discernment is given by 9 = {A, B, C}, and the power set is given 
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by 2 e = {0, A, B, A U B, C, A U C, B U C, A U B U C}. Hence we can define our 
first model Mi for our application by: 

if the expert says A: 
f m(A) = c A , 
\ m(AUBUC) = l-c A , 

if the expert says B: 

f m(B) = c B , (16) 
\ ro(iUBuC) = l-c B , 

if the expert says C: 
f m(C) = pa-ca+Pb-cb, 
\ m{AuBuC) = l-{ P A.CA+PB-c B )- 

On our numerical example, we obtain: 





A 




c 


iUBUC 


mi 


0.6 








0.4 


m 2 








0.5 


0.5 



Hence for the consensus combination for the model M\, the belief function mi2, the 
credibility, the plausibility and the pignistic probability are given by: 



element 


m 12 


bel 


pi 


betP 





0.3 










A 


0.3 


0.3 


0.5 


0.5238 


B 








0.2 


0.0952 


AUB 





0.3 


0.5 


0.6190 


C 


0.2 


0.2 


0.4 


0.3810 


AUC 





0.5 


0.7 


0.9048 


BUC 





0.2 


0.4 


0.4762 


AUBUC 


0.2 


0.7 


0.7 


1 



Where: 

mi 2 (0) - m 12 (inC) = 0.30. (17) 

This belief function provides an ambiguity because the same mass is put on A, the 
rock, and 0, the conflict. With the maximum of credibility, plausibility or pignistic 
probability this ambiguity is suppressed because these functions do not consider the 
empty set. 
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Model M 2 

In the first model Mi, the possible heterogeneity of the tile is taken into account. 
However, the ignorance is characterized by A U B U C and not by A U B anymore, and 
class C represents the situation when the two classes A and B are on X. Consequently 
Al) BUC could be equal to A U B, and we can propose another model M 2 given by: 

if the expert says A: 
( m(A) = c A , 
\ m(A U B) = 1 - ca, 

if the expert says B: 

f m{B) = c B , (18) 
\ m(AUB) = 1 - c B , 

if the expert says C: 
( m (C) = pa-ca+Pb-Cb, 
\ m{AuB) = 1 - (pa-ca +Pb-c b )- 

On our numerical example, we have: 





A 


B 


c 


Al) B 


mi 


0.6 








0.4 


m 2 








0.5 


0.5 



In this model M 2 the ignorance is partial and the conjunctive consensus rule, the 
credibility, the plausibility and the pignistic probability are given by: 



element 


m 12 


bcl 


pl 


betP 





0.5 










A 


0.3 


0.3 


0.3 


0.6 


B 


0.2 


0.2 


0.2 


0.4 


AUB 





0.5 


0.5 


1 


C 














AUC 





0.3 


0.3 


0.6 


BUC 





0.2 


0.2 


0.4 


AUBUC 





0.5 


0.5 


1 



where 

mi 2 (0) = m 12 (A n C) + m 12 {C flfiU B)) = 0.30 + 0.2 = 0.5. (19) 

The previous ambiguity in M\ between A (the rock) and (the conflict) is still 
present with a belief on higher than A. Moreover, in this model the mass on C is 
null! 
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These models Mi and Mq, are different because in the DST the classes A, B and C 
are supposed to be exclusive. Indeed, the fact that the power set 2 e is not closed under 
(~i operator leads to the exclusivity of the classes. 



Model M 3 

In our application, A, B and C cannot be considered exclusive on X. In order to 
propose a model following the DST, we have to study exclusive classes only. Hence, 
in our application, we can consider a space of discernment of three exclusive classes 
9 = {A n B c , B n A c , A n B} = {A 1 , B', C'}, following the notations given on the 
figure |2] 



.1 




Figure 2: Notation of the intersection of two classes A and B. 



Hence, we can propose a new model M3 given by: 



if the expert says A: 



f m(A'uC') = c A , 
\ m(A' UB'U C) -- 



1 



if the expert says B: 




1 



(20) 



if the expert says C: 
( m(C) = pa-ca + Pb-Cb, 
\ m(A' UB'UC') = 1- (pa-ca+Pb-cb)- 
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Note that A' U B' U C = A U B. On our numerical example we obtain: 





A'UC' 


B'UC 


a 


i'us'u a 


mi 


0.6 








0.4 


m 2 








0.5 


0.5 



Hence, the conjunctive consensus rule, the credibility, the plausibility and the pig- 
nistic probability are given by: 



element 


TO12 


bcl 


pl 


betP 
















A 1 = A n B c 








0.5 


0.2167 


B' = BC\A C 








0.2 


0.0667 


A' U B' = (A n s c ) u(Bn A c ) 








0.5 


0.2833 


C" = i4 n B 


0.5 


0.5 


1 


0.7167 


A'UC' = A 


0.3 


0.8 


1 


0.9333 


B'UC" = B 





0.5 


1 


0.7833 


4'UB'UC' = 4UB 


0.2 


1 


1 


1 



where 

m 12 (C) = m 12 (A n B) = 0.2 + 0.3 = 0.5. (21) 

On this example, with this model M3 the decision will be A with the maximum of 
pignistic probability. But the decision could a priori be taken also on C — A n B 
because mi 2 (C") is the highest. We show however in the discussion section that it is 
not possible. 

Model M A 

In the context of the DSmT, we can write C = AC\ B and easily propose a fourth 
model M4, without any consideration on the exclusivity of the classes, given by: 

if the expert says A: 
( m(A) = c A , 
\ m(A U B) = 1-ca, 

if the expert says B: 

f m{B) - c B , (22) 
\ m(A U B) = l-c B , 

if the expert says An B: 
( m(AnB) =p A .c A +p B -CB, 
\ m(A UB) = 1- (pa-ca + Pb-Cb)- 
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This last model M4 allows to represent our problem without adding an artificial class 
C. Thus, the model M4 based on the DSmT gives: 





A 


B 


Af)B 


AUB 


mi 


0.6 








0.4 


TO 2 








0.5 


0.5 



The obtained mass mi2 with the conjunctive consensus yields: 

m 12 (A) = 0.30, 
m 12 (B) = 0, 

m 12 {A n B) = mi(A)m,2(A nfl) + mi(A U B)m 2 {A n B) (23) 

= 0.30 + 0.20 = 0.5, 
mi 2 (AUB) = 0.20. 

These results are exactly the same for the model M3. These two models do not 
present ambiguity and show that the mass onAflB (rock and sand) is the highest. 

The generalized credibility, the generalized plausibility and the generalized pignis- 
tic probability are given by: 



element 


m 12 


Bel 


PI 


GPT 
















A 


0.3 


0.8 


1 


0.9333 


B 





0.5 


0.7 


0.7833 


ADB 


0.5 


0.5 


1 


0.7167 


AUB 


0.2 


1 


1 


1 



Like the model M3, on this example, the decision will be A with the maximum 
of pignistic probability criteria. But here also the maximum of m\ 2 is reached for 

A n b = a. 

If we want to consider only the kind of possible sediments A and B and not also the 
conjunctions, we can use a proportional conflict redistribution rules such as the PCR5 
proposed in IH31 . Consequently we have x = 0.3.(0.5/0.3) = 0.5 and y = 0, and the 
PCR5 rule provides: 

mp C R 5 {A) = 0.30 + 0.5 = 0.8, 

mpcR 5 (B) = 0, (24) 
mp C R 5 (A Ufl) =0.20. 



Arnaud MARTIN and Christophe OSSWALD 



13 



The credibility, the plausibility and the pignistic probability are given by: 



element 




bcl 


pl 


bctP 
















A 


0.8 


0.8 


1 


0.9 


B 








0.2 


0.1 


AUB 


0.2 


1 


1 


1 



On this numerical example, the decision will be the same than the consensus rule, here 
the maximum of pignistic probability is reached for A (rock). In the next section we 
see that is not always the case. 

Model M 5 

Another model M 5 which can be used in both the DST and the DSmT is given consid- 
ering only one belief function according to the proportion by: 

m{A) = pa-ca, 

m(B) = pb-cb, (25) 
m(A U B) = 1 - (pa-ca +Pb-cb)- 

If for one expert, the tile contains only A, pa = 1, and m(B) = 0. If for another 
expert, the tile contains A and B, we take into account the certainty and proportion of 
the two sediments but not only on one focal element. Consequently, we have simply: 





A 


B 


AUB 


mi 


0.6 





0.4 


7712 


0.3 


0.2 


0.5 



In the DST context, the consensus rule, the credibility, the plausibility and the 
pignistic probability are given by: 



element 


mu 


bel 


pl 


bctP 





0.12 










A 


0.6 


0.6 


0.8 


0.7955 


B 


0.08 


0.08 


0.28 


0.2045 


AUB 


0.2 


0.88 


0.88 


1 



In this case we do not have the plausibility to decide on AD B, because the conflict is 
on 0. 
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In the DSmT context, the consensus rule, the generalized credibility, the general- 
ized plausibility and the generalized pignistic probability are given by: 



element 


m 12 


Bel 


PI 


GPT 
















A 


0.6 


0.72 


0.92 


0.8933 


B 


0.08 


0.2 


0.4 


0.6333 


AnB 


0.12 


0.12 


1 


0.5267 


AUB 


0.2 


1 


1 


1 



The decision with the maximum of pignistic probability criteria is still A. 
The PCR5 rule provides: 



element 


mpcR5 


bcl 


pl 


bctP 
















A 


0.69 


0.69 


0.89 


0.79 


B 


0.11 


0.11 


0.31 


0.21 


AUB 


0.2 


1 


1 


1 



where 

mp C R 5 (A) = 0.60 + 0.09 = 0.69, 
mpcR 5 (B) = 0.08 + 0.03 = 0.11. 

With this model and example the PCR5 rule, the decision will be also A, and we do 
not have difference between the consensus rules in the DST and DSmT. 

2 Discussion 

We have build, in the previous section, the models Mi, M 2 , M 3 , M4, and M 5 in the 
DSmT case in order to take into account the decision considering also An B ("there 
is rock and sand on the tile"). In fact only the Mi and M 2 models can do it. Model 
M 2 can do it only if both experts say A Pi B. These two models assume that A, B 
and A n B are exclusive. Of course this assumption is false. For the models M3, M4 
and M 5 , we have to take the decision on the credibilities, plausibilities or pignistic 
probabilities, but these three functions for A n B cannot be higher than A or B (or 
for C than A' U C and B' U C with the notations of the model M 3 ). Indeed for all 
x e A n B, x e A and x £ B, so for all X C Y: 

belpf) < bel(F), 

pipo < P i(n 

betPpf) < betP(F), 
Bel(X) < Bel(Y), 
P1P0 < P1(Y), 
GPT(X) < GPT(F). 
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Hence, our first problem is not solved: we can never choose An B with the max- 
imum of credibility, plausibility or pignistic probability. If the two experts think that 
the considered tile contains rock and sand (A n B), then the pignistic probabilities are 
equal. However the belief onAnB can be the highest (see the example on the models 
M 3 and M 4 ). The limits of the decision rules are reached in this case. 

We have seen that we can describe our problem both in the DST and the DSmT 
context. The DSmT is more adapted to modelize the belief onAflB for example with 
the model M 4 , but model M 3 with the DST can provide exactly the same belief on A, 
B and An B. Consequently, the only difference we can expect on the decision comes 
from the combination rules. In the presented numerical example, the decisions are the 
same: we choose A. 

An example of decision instability 

Take another example with this last model M 5 : The first expert provides: pa = 0.5, 
Pb = 0.5, ca = 0.6 and cb = 0.4, and the second expert provides: pa = 0.5, 
Pb = 0.5, ca = 0.86 and cs = 1. We want take a decision only on A or B. Hence we 
have: 





A 


B 


AUB 


mi 


0.3 


0.2 


0.5 


Wi2 


0.43 


0.5 


0.07 



For M5 on the DST context: 



element 


m 12 


bel 


pl 


betP 





0.236 










A 


0.365 


0.365 


0.4 


0.5007 


B 


0.364 


0.364 


0.399 


0.4993 


AUB 


0.035 


0.764 


0.764 


1 



M 5 with PCR5 gives (with the partial conflicts: x\ = 0.0562, y x = 0.0937, x 2 = 
0.0587 and y 2 = 0.0937): 



element 


m PC R5 


bel 


pl 


bctP 
















A 


0.479948 


0.479 


0.5149 


0.4974 


B 


0.485052 


0.485 


0.5202 


0.5026 


AUB 


0.035 


1 


1 


1 



This last example shows that we have a difference between the DST and the DSmT, but 
what is the best solution? With the DST we choose A and with the DSmT we choose 
B. We can show that the decision will be the same in the most of the case (about 
99.4%). 
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Stability of decision process 

The space where experts can define their opinions on which n classes are present in a 
given tile is a part of [0, l] n : £ = [0, 1]" n ( S~] m(X) < 1). In order to study the 

xee 

different combination rules, and the situations where they differ, we use a Monte Carlo 
method, considering the weights pa, ca, Pb, cb, ■ ■ ■ , as uniform variables, filtering 
them by the condition pxcx < 1 for one expert. 

xee 

Thus, we measure the proportion of situations where decision differs between 
the consensus combination rule, and the PCR5, where conflict is proportionally dis- 
tributed. 

We can not choose A n B, as the measure of A n B is always lower (or equal with 
probability 0) than the measure of A or B. In the case of two classes, A U B is the 
ignorance, and is usually excluded (as it always maximises bel, pi, betP, Bel, PI and 
GPT). We restrict the possible choices to singletons, A, B, etc. Therefore, it is equiv- 
alent to tag the tile by the most credible class (maximal for bel), the most plausible 
(maximal for pi), the most probable (maximal for betP) or the heaviest (maximal for 
m), as the only focal elements are singletons, and 0. 

The only situation where the total order induced by the masses m on singletons can 
be modified is when the conflict is distributed on the singletons, as is the case in the 
PCR5 method. 

Thus, for two classes, the subspace where the decision is "rock" by consensus rule 
is very similar to the subspace where the decision is "rock" by the PCR5 rule: only 
0.6% of the volume differ. For a higher number of classes, the decision obtained by 
fusing the two experts' opinions is much less stable: 



number of classes 


2 


3 


4 


5 


6 


7 


decision change 


0.6% 


5.5% 


9.1% 


12.1% 


14.6% 


16.4% 



Therefore, the specificity of PCR5 appears mostly with more than two classes, 
and the different combination rules are nearly equivalent when decision must be taken 
within two possible classes. 

Left part of figure[3]shows the density of conflict within £ , for a number of classes 
of 2, 3, 6 and 7. Right part shows how this distribution changes if we restrict £ to the 
cases where the decision changes between consensus (dotted lines) and PCR5 (plain 
lines). Conflict is more important in this subspace, mostly because a low conflict usu- 
ally means a clear decision: the measure on the best class is often very different than 
measure on the second best class. 

For the "two experts and two classes" case, it is difficult to characterize analytically 
the stability of the decision process. However, we can easily show that if mi(A) = 
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0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 



Figure 3: Density of conflict for (left) uniform random experts and (right) data with 
different decision between consensus and PCR5. 

m 2 (B) or if mi (A) = mi(B), the final decision does not depend on the chosen 
combination rule. 

3 Illustration 

Database 

Our database contains 40 sonar images provided by the GESMA (Groupe 
d'Etudes Sous-Marines de l'Atlantique). These images were obtained with a Klein 
5400 lateral sonar with a resolution of 20 to 30 cm in azimuth and 3 cm in range. The 
sea-bottom depth was between 15 m and 40 m. 

Two experts have manually segmented these images giving the kind of sediment 
(rock, cobble, sand, silt, ripple (horizontal, vertical or at 45 degrees)), shadow or other 
(typically ships) parts on images, helped by the manual segmentation interface pre- 
sented in figure|4] All sediments are given with a certainty level (sure, moderately sure 
or not sure). Hence, every pixel of every image is labeled as being either a certain type 
of sediment or a shadow or other. 

Results 

We note A — rock, B = cobble, C = sand, D = silt, E = ripple, F = shadow and 
G = other, hence we have seven classes and = {A, B, C, D, E, F, G}. We have 
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« "s 5W ra 



Figure 4: Manual Segmentation Interface. 

applied the generalized model M 5 on tiles of size 32 x 32 given by: 

m(A) =p A1 .ci +PA2-C2 +PA3-C3, for rock, 
m(B) =Pbi-Ci +PB2-C2 +PB3-C3, for cobble, 
m(C) =pci-c\ +PC2-C2 +PC3-C3, for ripple, 
m(D) =pdx-c\ +PD2-C2 +PD3-C3, for sand, 
m(E) = pe\-C\ +PE2-C2 +PE3-C3, for silt, 
m(F) =p F1 .c 1 +PF2-C2 +PF3-C3, for shadow, 
m(G) = pai-Ci +PG2-C2 +PG3-C3, for other, 

m(0) = 1 - (m(A) + m(B) + m(C) + m(D) + m{E) + m(F) + m(G)), 

where c\, C2 and C3 are the weights associated to the certitude respectively: "sure", 
"moderately sure" and "not sure". The chosen weights are here: c\ = 2/3, C2 = 
1/2 and C3 = 1/3. Indeed we have to consider the cases when the same kind of 
sediment (but with different certainties) is present on the same tile. The proportion 
of each sediment in the tile associated to these weights is noted, for instance for A: 
PAi, PA2 and pA3- The tableQ]gives the conflict matrix of the two experts. We note 
that the most of conflict come from a difference of opinion between sand and silt. For 
instance, the expert 1 provides many tiles of sand when the expert 2 thinks that is silt 
(conflict induced of 0.0524). This conflict is explained by the difficulty for the experts 
to differentiate sand and silt that differ with only the intensity. Part of conflict comes 
also from the fact that ripples are hard to distinguish from sand or silt. Ripples, that is, 
sand or silt in a special configuration, is sometimes difficult to see on the images, and 
the ripples are most of the time visible in a global zone where sand or silt is present. 
Cobbles also yield conflicts, especially with sand, silt and rock: cobble is described by 
some small rocks on sand or silt. The total conflict between the two experts is 0.1209. 
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Expert 2 





Rock 


Cobble 


Ripple 


Sand 


Silt 


Shadow 


Other 


Rock 




12.87 


2.72 


4.42 


3.91 


6.41 


0.22 


Cobble 


5.59 




0.85 


18.44 


3.85 


0.04 





Ripple 


3.12 


3.38 




30.73 


150.60 


0.27 


0.16 


Sand 


9.50 


43.39 


42.60 




524.33 


0.51 


0.57 


Silt 


6.42 


27.05 


36.22 


258.98 




2.60 


0.11 


Shadow 


3.82 


0.15 


2.13 


1.38 


0.50 




0.41 


Other 





0.20 


0.10 


0.35 


0.31 


0.14 





Table 1 : Matrix of conflict ( x 10 4 ) between the two experts. 



Hence, our application does not present a large conflict. 

We have applied the consensus rule and the PCR5 rule with this model. The de- 
cision is given by the maximum of pignistic probability. In most of the cases the 
decisions taken by the two rules are the same. We note a difference only on 0.4657% 
of the tiles. Indeed, we are in the seven classes case with only 0.1209 of conflict, the 
simulation given on the figure 3 show that we have few chance that the decisions differ. 



4 Conclusion 

In this paper we have proposed five different models in order to take into account 
two classical problems in uncertain image classification (for training or evaluation): 
the heterogeneity of the considered tiles and the certainty of the experts. These five 
models have been developed in the DST and DSmT contexts. The heterogeneity of 
the tile and the certainty of the expert can be easily taken into account in the models. 
However, if we want to have the plausibility of taking a decision on such a tile (with a 
conjunction Ap\ B) the usual decision functions (credibility, plausibility and pignistic 
probability) are not sufficient: they cannot allow a such decision. We can take the 
decision oniflB only if we consider the belief function and if the model provides a 
belief oniflB. 

We have also studied the decision according to the conflict and to the combination 
rules: conjunctive consensus rule and PCR5 rule. The decision (taken with the maxi- 
mum of the credibility, the plausibility or the pignistic probability) is the same in most 
of the cases. For two experts, more classes leads to more conflict and to more cases 
giving a different decision with the different rules. 

We have also illustrated one of the proposed models on real sonar images classified 
manually by two different experts. In this application the total conflict between the two 
experts is 0. 1209 and we note a difference of decision only on 0.4657% of the tiles. 
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We can easily generalize our models for three or more experts and use the general- 
ized combination of the PCR5 given by the equation (|7). Of course the conflict will be 
higher and the difference in the decision must be studied. 
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